Classifier Fitness Based on Accuracy Stewart

In many classifier systems, the classifier strength parameter serves as a predictor of future payoff and as the classifier's fitness for the genetic algorithm. We investigate a classifier system, XCS, in which each classifier maintains a prediction of expected payoff, but the classifier's fitness is given by a measure of the prediction's accuracy. T h e system executes the genetic algorithm in niches defined by the match sets, instead of panmictically. These aspects of XCS result in its population tending to form a complete and accurate mapping X x A + P from inputs and actions to payoff predictions. Further, XCS tends to evolve classifiers that are maximally general, subject to an accuracy criterion. Besides introducing a new direction for classifier system research, these properties of XCS make it suitable for a wide range of reinforcement learning situations where generalization over states is desirable. Traditionally in classifier systems, the classifier strength parameter serves both as a predictor of future payoff and as the classifier's fitness for the genetic algorithm (GA). However, predicted payoff may inadequately represent fitness. For example, a low-predicting classifier may nevertheless be the best one for its environmental niche. We investigate a classifier system, XCS, in which each classifier maintains a prediction of expected payoff, but the classifier's fitness is not given by the prediction. Instead, the fitness is a separate number based on an inverse function of the classifier's average prediction error; that is, it is based on a measure of the accuracy of the prediction, instead of the prediction itself. XCS also executes the genetic algorithm in niches defined by the match sets (Booker, 1982) rather than panmicucally. The present research-an investigation into classifier system technique-stemmed from dissatisfaction with the behavior of traditional classifier systems, and the hypothesis that the shortcomings were due in part to the definition of fitness. As we will discuss in Section 5.1, some previous work had factored measures of accuracy into the fitness function. However, the results with XCS show that a complete shift to accuracy-based fitness is not only possible, but yields a classifier system that is superior to traditional systems in important respects. Specifically, accuracy-based fitness, in combination with a niche GA, results in XCS's population tending to form a complete and accurate mapping X x A =+ P from inputs and actions to payoff predictions. Traditional classifier systems have not theoretically emphasized or actually produced such mappings, …